WHAT IS CLAIMED IS; 

1. An object-based three-dimensional (3-D) audio server system 
comprising: 

an audio input unit receiving object-based sound sources through 
various input devices; 

an audio editing/producing unit separating the sound sources applied 
through the audio input unit into object sounds and background sounds 
according to a user's selection, and converting them into 3-D audio scene 
information; and 

an audio encoding unit encoding 3-D information and object signals of 
the 3-D audio scene information converted by the audio editing/producing unit 
so as to transmit them through a medium. 

2. The system according to claim 1, wherein sound sources selected by 
the user from among the sound sources that have been applied through the 
audio input unit are processed into object sounds, and other sound sources not 
selected by the user are processed into background sounds. 

3. The system according to claim 1, wherein the audio input unit 
includes: 

a combination of sound source input devices having: 

a single channel microphone with a single microphone; 

a stereo microphone with at least two microphones; 

a dummy head microphone whose shape is like a head of a 

human body; 

an ambisonic microphone receiving the sound sources after 
dividing them into signals and volume levels, each moving with a given 
trajectory on 3-D X, Y, and Z coordinates; and 

a multi-channel microphone receiving multitrack audio signals ; 

and 

a source separation/3-D information extractor separating the sound 
sources applied from the combination of the sound source input devices by 
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objects, and extracting 3-D information. 

4. The system according to claim 1, wherein the audio editing/producing 
unit includes: 

a router/audio mixer dividing the sound sources applied in the multi- 
track format into a plurality of sound source objects and background sounds; 

a scene editor/producer editing an audio scene and producing the 
edited audio scene by using 3-D information and spatial information of the 
sound source objects and background sound objects divided by the router/audio 
mixer; and 

N a controller providing a user interface so that the scene editor/producer 
edits an audio scene and produces the edited audio scene under the control of 
a user. 

5. The system according to claim 1, wherein the audio encoding unit 
includes: 

a data encoding block encoding each set of data divided into 
background sound objects, sound source objects, and audio scene information 
output from the audio editing/producing unit; and 

a multiplexer multiplexing object data of the background sound, data of 
the sound sources, and data of the audio scene information encoded by the 
data encoding block into a single signal, and transmitting the same. 

6. The system according to claim 5, wherein the data decoding block 
includes: 

an audio object encoder encoding the sound objects; 
an audio scene information encoder encoding the audio scene 
information; and 

a background sound object encoder encoding the background sounds. 

7. A method of controlling an object-based 3-D audio server system 
comprising: 

separating sound source objects from among sound sources according 
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to a selection by a user; 

inputting 3-D information for each sound source object separated from 
the applied sound sources; 

mixing sound sources other than the separated sound source objects 
into background sounds; and 

forming the sound source objects, the 3-D information, and the 
background sound objects into an audio scene, and encoding and multiplexing 
the audio>scene to transmit the encoded and multiplexed audio signal through a 
medium. 

8. The method according to claim 7, wherein each of the sound source 
objects further includes 3-D information for a relative sound source object by 
grouping the sound source objects that have to be controlled by-groups. 

9. An object-based three-dimensional audio terminal system 
comprising: 

an audio decoding unit demultiplexing and decoding a multiplexed 
audio signal including object sounds, background sounds, and scene 
information applied through a medium; 

an audio scene-synthesizing unit selectively synthesizing the object 
sounds with the audio scene information decoded by the audio decoding unit 
into a 3-D audio scene under the control of a user; 

a user control unit providing a user interface so as to selectively 
synthesize the audio scene by the audio scene synthesizing unit under the 
control of the user; and 

an audio reproducing unit reproducing the 3-D audio scene synthesized 
by the audio scene-synthesizing unit. 

10. The system according to claim 9, wherein the audio decoding unit 
includes: 

a demultiplexer demultiplexing the data applied through the medium 
and multiplexed to separate them into background sound object data, sound 
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source data, and audio scene information data; and 

a decoder decoding the background sound object data, the sound 
source data, and the audio scene information data separated by the 
demultiplexer. 

11. The system according to claim 9, wherein the audio scene- 
synthesizing unit includes: 

a sound source object processor receiving the background sound 
objects, the sound source objects, and the audio scene information decoded by 
the audio decoding unit to process the sound source objects and audio scene 
information according to a motion, a relative location between the sound source 
objects, and a three-dimensional location of the sound source objects, and 
spatial characteristics under the control of the user; and 

an object mixer mixing the sound source objects processed by the 
sound source object processor with the background sound objects decoded by 
the audio decoding unit to output results. 

12. The system according to claim 9, wherein the sound source object 
processor further includes: 

a motion processor analyzing a plurality of sound source data and the 
audio scene information, calculating a location of each sound source object 
moving with its particular trajectory, and modifying its trajectory under the 
control of the user through the user control unit; 

a group object processor calculating a relative location of the respective 
sound source objects when a plurality of the sound source objects is grouped, 
and controlling the relative location of the sound source objects under the 
control of the user through the user control unit; 

a 3-D sound localization processor providing each sound source object 
having a location defined on 3-D coordinates with directivity in response to a 
listener's location under the control of the user control unit; and 

a 3-D space modeling processor providing a sense of closeness and 
remoteness and spatial effects to each sound source object according to 
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characteristics of a 3-D space. 

13. The system according to claim 9, wherein the audio reproducing 
unit includes: 

an acoustic environment equalizer equalizing the acoustic environment 
between a listener and a reproduction system in order to accurately reproduce 
the 3-D audio transmitted from the audio scene synthesizing unit; 

an acoustic environment corrector calculating a coefficient of a filter for 
the acoustic environment equalizer's equalization, and correcting the 
equalization by the user; and 

an audio signal output device outputting a 3-D audio signal equalized by 
the acoustic environment equalizer. 

14. The system according to claim 9, wherein the acoustic environment 
equalizer further includes: 

means for equalizing the environmental characteristics between the 
listener and the audio terminal system in order to accurately reproduce 3-D 
audio; 

means for canceling crosstalk transmitted to right and left ears of the 
listener; and 

means for correcting the characteristics of the acoustic environment 
automatically or in response to the user's input, according to the information on 
speakers of the audio system, a listening room's construction, and arrangement 
of the speakers, transmitted from the acoustic environment corrector. 

15. The system according to claim 9, wherein the user control unit 
includes an interface that controls each sound source object and the listener's 
direction and position, and receives the user's control for maintaining realism of 
sound reproduction in a virtual space to transmit a control signal to each unit. 

16. A method of controlling an object-based 3-D audio terminal system 
comprising: 
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in receiving and outputting an object-based 3-D audio signal, decoding 
the audio signal applied through a medium, and dividing the audio signal into 
object sounds, 3-D information, and background sounds; 

performing motion processing, group object processing, 3-D sound 
localization, and 3-D space modeling on the object sounds and the 3-D 
information to modify and apply the processed object sounds and 3-D 
information according to a user's selection, and mixing them with the 
background sounds; and 

equalizing the mixed audio signal in response to correction of 
characteristics of the acoustic environment that the user controls, and outputting 
the equalized signal. 

17. The method according to claim 16, wherein synthesizing the audio 
scene further includes: 

processing a motion effect of each object moving with a particular 
trajectory, in response to a control signal output from a user control unit; 

grouping the object, and calculating and processing a relative location 
of each grouped object; 

processing 3-D sound localization by providing each sound source 
object having a location defined on 3-D coordinates with directivity in response 
to a listener's position; 

processing 3-D space modeling by providing the object with a sense of 
closeness and remoteness and spatial effects according to characteristics of a 
3-D space; and 

mixing the processed sound source object with the background sound 
object to synthesize a 3-D audio scene. 

18. The method according to claim 16, wherein outputting the audio 
scene further includes: 

equalizing the 3-D audio output according to information on 
characteristics of the acoustic environment between a listener and the audio 
system, and information on correcting the acoustic environment applied by the 
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user; and 

outputting the equalized 3-D audio scene to provide the same to the 

listener. 

19. An object-based three-dimensional audio system comprising: 

an audio input unit receiving object-based sound sources through input 

devices; 

an audio editing/producing unit separating the sound sources applied 
through the audio input unit into object sounds and background sounds 
according to a user's selection, and converting them into three-dimensional 
audio objects; 

an audio encoding unit encoding 3-D information of the audio objects 
and object signals converted by the audio editing/producing unit to transmit 
them through a medium; 

an audio decoding unit receiving the audio signal including object 
sounds and 3-D information encoded by the audio encoding unit through the 
medium, and decoding the audio signal; 

an audio scene synthesizing unit selectively synthesizing the object 
sounds with 3-D information decoded by the audio decoding unit into a 3-D 
audio scene under the control of a user; 

a user control unit outputting a control signal according to the user's 
selection so as to selectively synthesize the audio scene by the audio scene 
synthesizing unit under the control of the user; and 

an audio reproducing unit reproducing the audio scene synthesized by 
the audio scene synthesizing unit. 

20. A method of controlling an object-based 3-D audio terminal system, 
comprising: 

separating sound source objects from among sound sources according 
to a selection by a user; 

inputting 3-D information on the separated sound source objects; 
processing sound sources other than the input sound source objects 
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and 3-D information as background sounds; 

forming the sound source objects, the 3-D information, and the 
background sounds into an audio scene, and encoding and multiplexing the 
audio scene to transmit the encoded and multiplexed audio scene through a 
medium; 

decoding the audio signal applied through a medium, and dividing the 
audio signal into object sounds, 3-D information, and background sounds; 

performing motion processing, group object processing, 3-D sound 
localization, and 3-D space modeling with respect to the object sounds and the 
3-D information to modify and apply the processed object sounds and- 3-D 
information according to a user's selection, and mixing them with the 
background sounds; and 

equalizing the mixed audio signal in response to correction of 
characteristics of the acoustic environment that the user controls, and outputting 
the equalized audio signal. . 
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